Skip to content

HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile#3722

Merged
jojochuang merged 5 commits intoapache:trunkfrom
virajjasani:HADOOP-18025-trunk
Dec 2, 2021
Merged

HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile#3722
jojochuang merged 5 commits intoapache:trunkfrom
virajjasani:HADOOP-18025-trunk

Conversation

@virajjasani
Copy link
Contributor

@virajjasani virajjasani commented Nov 25, 2021 •
edited
Loading

Description of PR

Upgrade HBase version to 1.7.1 for hbase1 profile. The current version is quite old.
Also, replace old deprecated APIs of Scan with better alternatives.

How was this patch tested?

Verified with some local testing. All CRUD unit tests are tested.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@virajjasani
Copy link
Contributor Author

virajjasani commented Nov 25, 2021

TimelineService tests will pass after PR #3712 is merged.

@virajjasani virajjasani mentioned this pull request Nov 26, 2021
Merged
1 task
jojochuang
jojochuang reviewed Nov 29, 2021
View reviewed changes
...main/java/org/apache/hadoop/yarn/server/timelineservice/storage/reader/EntityTypeReader.java Outdated
Scan scan = new Scan(startPrefix, endPrefix);
Scan scan = new Scan();
scan.withStartRow(startPrefix);
scan.withStopRow(endPrefix);
Copy link
Contributor

@jojochuang jojochuang Nov 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it would be the best it make it as the following:
scan.withStartRow()
.withStopRow()
.setFilter()
.setSmall()

jojochuang
jojochuang approved these changes Nov 29, 2021
View reviewed changes
Copy link
Contributor

@jojochuang jojochuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@hadoop-yetus
Copy link

hadoop-yetus commented Nov 30, 2021

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 8s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 shelldocs 0m 1s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ pathlen 0m 0s /results-pathlen.txt The patch appears to contain 3 files with names longer than 240
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 12m 38s Maven dependency ordering for branch
+1 💚 mvninstall 21m 36s trunk passed
+1 💚 compile 21m 58s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 19m 23s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 3m 46s trunk passed
+1 💚 mvnsite 26m 32s trunk passed
+1 💚 javadoc 8m 8s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 8m 26s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗 spotbugs 0m 18s branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚 shadedclient 54m 46s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 53s Maven dependency ordering for patch
+1 💚 mvninstall 22m 53s the patch passed
+1 💚 compile 22m 13s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 22m 13s root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 0 new + 1916 unchanged - 13 fixed = 1916 total (was 1929)
+1 💚 compile 19m 14s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 19m 14s root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu120.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu120.04-b10 generated 0 new + 1795 unchanged - 13 fixed = 1795 total (was 1808)
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 3m 35s the patch passed
+1 💚 mvnsite 21m 47s the patch passed
+1 💚 shellcheck 0m 0s No new issues.
+1 💚 xml 0m 2s The patch has no ill-formed XML file.
+1 💚 javadoc 7m 47s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 7m 57s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗 spotbugs 0m 20s hadoop-project has no data from spotbugs
+1 💚 shadedclient 55m 49s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 765m 43s /patch-unit-root.txt root in the patch passed.
+1 💚 asflicense 1m 39s The patch does not generate ASF License warnings.
1117m 0s
Reason Tests
Failed junit tests hadoop.yarn.csi.client.TestCsiClient
hadoop.tools.dynamometer.TestDynamometerInfra
hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/8/artifact/out/Dockerfile
GITHUB PR #3722
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle shellcheck shelldocs
uname Linux 5850570d138a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 476b777
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/8/testReport/
Max. process+thread count 2939 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/8/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

hadoop-yetus commented Nov 30, 2021

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 37s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 shelldocs 0m 1s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ pathlen 0m 0s /results-pathlen.txt The patch appears to contain 3 files with names longer than 240
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 12m 42s Maven dependency ordering for branch
+1 💚 mvninstall 22m 6s trunk passed
+1 💚 compile 22m 5s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 19m 19s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 3m 40s trunk passed
+1 💚 mvnsite 25m 19s trunk passed
+1 💚 javadoc 8m 1s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 8m 14s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗 spotbugs 0m 19s branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚 shadedclient 53m 56s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 55s Maven dependency ordering for patch
+1 💚 mvninstall 22m 15s the patch passed
+1 💚 compile 21m 29s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 21m 29s root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 0 new + 1917 unchanged - 13 fixed = 1917 total (was 1930)
+1 💚 compile 19m 8s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 19m 8s root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu120.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu120.04-b10 generated 0 new + 1791 unchanged - 13 fixed = 1791 total (was 1804)
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 3m 33s the patch passed
+1 💚 mvnsite 21m 31s the patch passed
+1 💚 shellcheck 0m 0s No new issues.
+1 💚 xml 0m 2s The patch has no ill-formed XML file.
+1 💚 javadoc 7m 49s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 8m 3s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗 spotbugs 0m 21s hadoop-project has no data from spotbugs
+1 💚 shadedclient 54m 18s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 795m 50s /patch-unit-root.txt root in the patch passed.
+1 💚 asflicense 1m 35s The patch does not generate ASF License warnings.
1141m 21s
Reason Tests
Failed junit tests hadoop.yarn.csi.client.TestCsiClient
hadoop.tools.dynamometer.TestDynamometerInfra
hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/9/artifact/out/Dockerfile
GITHUB PR #3722
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle shellcheck shelldocs
uname Linux 11794c51c104 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 476b777
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/9/testReport/
Max. process+thread count 3426 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/9/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

hadoop-yetus commented Nov 30, 2021

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 52s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 shelldocs 0m 1s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ pathlen 0m 0s /results-pathlen.txt The patch appears to contain 3 files with names longer than 240
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 12m 27s Maven dependency ordering for branch
+1 💚 mvninstall 24m 33s trunk passed
+1 💚 compile 25m 43s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 compile 22m 29s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 checkstyle 4m 19s trunk passed
+1 💚 mvnsite 28m 7s trunk passed
+1 💚 javadoc 9m 2s trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 10m 57s trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗 spotbugs 0m 16s branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚 shadedclient 61m 45s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 50s Maven dependency ordering for patch
+1 💚 mvninstall 26m 56s the patch passed
+1 💚 compile 24m 11s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javac 24m 11s root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 0 new + 1921 unchanged - 13 fixed = 1921 total (was 1934)
+1 💚 compile 22m 9s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚 javac 22m 9s root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu120.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu120.04-b10 generated 0 new + 1796 unchanged - 13 fixed = 1796 total (was 1809)
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 3m 54s the patch passed
+1 💚 mvnsite 25m 1s the patch passed
+1 💚 shellcheck 0m 0s No new issues.
+1 💚 xml 0m 3s The patch has no ill-formed XML file.
+1 💚 javadoc 9m 22s the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚 javadoc 8m 22s the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗 spotbugs 0m 15s hadoop-project has no data from spotbugs
+1 💚 shadedclient 62m 16s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 995m 23s /patch-unit-root.txt root in the patch passed.
+1 💚 asflicense 1m 37s The patch does not generate ASF License warnings.
1388m 46s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
hadoop.yarn.csi.client.TestCsiClient
hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure
hadoop.tools.dynamometer.TestDynamometerInfra
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/11/artifact/out/Dockerfile
GITHUB PR #3722
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle shellcheck shelldocs
uname Linux 431d216ad92f 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 476b777
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/11/testReport/
Max. process+thread count 2390 (vs. ulimit of 5500)
modules C: hadoop-project hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3722/11/console
versions git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@virajjasani
Copy link
Contributor Author

virajjasani commented Nov 30, 2021

Thanks for the review @jojochuang. The build has run for the entire project, all modules combined. Test failures are known (irrelevant to this PR) failures.

@jojochuang jojochuang merged commit b34dcb5 into apache:trunk Dec 2, 2021
@jojochuang
Copy link
Contributor

jojochuang commented Dec 2, 2021

Merged. Thanks @virajjasani !

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
@virajjasani
HADOOP-18025. Upgrade HBase version to 1.7.1 for hbase1 profile (apac…
858cddb 
…he#3722)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jojochuang jojochuang jojochuang approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@virajjasani @hadoop-yetus @jojochuang